Computer messaging services are known and can be built around various different design paradigms, e.g., in connection with various standards. One messaging paradigm is the publish/subscribe (or “pub/sub”) model, which sometimes is used in connection with a message broker. See, for example, U.S. Publication No. 2010/0333111. One such standard is the Java Message Service (JMS). Some JMS providers are developing client-side failover and load balancing functionality with their own proprietary implementation mechanisms, e.g., in efforts to providing high availability (HA) and scalability.
Unfortunately, exhaustively testing such mechanisms typically requires a deep understanding of the particular implementation, as well as programming skills. Part of the problem is believed to relate to the general lack of existing tools that simulate broker crashes or broker stop/start events, e.g., in connection with publish/subscribe messaging models that use a broker as a messaging infrastructure for routing messages across client applications. It therefore oftentimes is difficult to ensure that publish/subscribe models remain intact in the presence of forced and/or unforced errors such as, for example, broker crashes, broker stop/start events, network failures etc.
Some testing approaches/techniques for publish/subscriber models (and/or its variants) or for a distributed network generally involve monitoring, debugging, scalability analysis, and/or performance characteristic tracking (e.g., for latency, through-put, etc.)—and related report generation. For example:                U.S. Pat. No. 7,523,198, for example, involves an integrated testing approach for monitoring and analyzing the application and network layers' performance characteristics (including latency), which may help in improving the performance of publish/subscribe network system.        U.S. Pat. No. 8,020,037 tries to test failover and failback mechanisms in connection with a data storage approach keeps a duplicate set of data to be replaced with primary data.        U.S. Pat. No. 6,704,883 discusses testing approaches for distributed systems using a test controller that publishes the test execution scripts to subscribing test agents and collects the report for consolidation from the distributed test agents.        U.S. Pat. No. 8,010,325 involves performing a simulation to assess availability of a service in the event of various types of failures. A specification describes a behavior to be induced, and a formula under which availability is to be measured. An agent on a machine looks up a routine in a simulation library to induce the behavior on that machine, and data is gathered and reported.        
Unfortunately, even these approaches do not cover client application reactions for failover and performance optimization testing techniques by simulating abrupt system/application/message engine's (e.g., broker) shutdowns, crashes, and stop/start events in an automated fashion, e.g., in a distributed or cloud computing environment.
Thus, it will be appreciated that there is a need in the art for improved testing techniques that are capable of simulating problems, e.g., in connection with testing how a client responds to the same, e.g., in connection with a messaging (e.g., publish/subscribe) model and/or application integration system including such a messaging model.
One aspect of certain example embodiments relates to addressing the problem of testing a real customer client-side failover reaction in a wide range of scenarios including actual message deliverer (e.g., broker) crashes. It will be appreciated that this testing may be performed in place of, or together with, testing “regular events” using a test script. These example techniques may be used in connection with a wide range of different network environments including, for example, distributed networks, cloud computing environments, etc. In certain example embodiments, the message sending/receiving may be managed according to a “publish/subscribe via a message deliverer scenario.”
In accordance with certain example embodiments, a messaging system is provided. A broker cluster includes a plurality of brokers configured to relay messages from at least one publisher to at least one subscriber over one or more networked cluster connections in accordance with a predefined publish/subscribe model related policy. A test driver is configured to receive instructions regarding errors to be simulated in association with one or more components in the messaging system. A simulator is configured to simulate a network link and properties associated with the link. Processing resources include at least one processor and a memory configured to: (a) coordinate with the test driver and the simulator to selectively generate errors in one or more components of the messaging system, post-deployment and while it is live, in accordance with the script, and (b) determine whether the messaging system appropriately handled the errors selectively generated in response to the script.
In accordance with certain example embodiments, there is provided a testing system for use in a messaging system comprising a broker cluster including a plurality of brokers configured to relay messages from at least one publisher to at least one subscriber over one or more cluster connections in accordance with a predefined publish/subscribe model related policy. The testing system comprises a test driver configured to receive a script causing errors to be simulated in association with one or more components in the messaging system. A wide area network (WAN) (e.g., for use in a wireless network) simulator is configured to simulate a network link and properties associated with the link, with the properties including latency, jitter, bandwidth, and/or packet loss. Processing resources including at least one processor and a memory are configured to: (a) coordinate with the test driver and the simulator to selectively generate errors in one or more components of the messaging system, after the messaging system has been deployed and while it is live, in accordance with the script, and (b) determine whether the messaging system appropriately handled the errors selectively generated in response to the script. The messaging system is deployed in a distributed network or cloud computing environment, and the testing system is operable within that environment.
In accordance with certain example embodiments, there is provided a method of testing a messaging system comprising a broker cluster including a plurality of brokers configured to relay messages from at least one publisher to at least one subscriber over one or more cluster connections. Messages can be relayed from the at least one publisher to the at least one subscriber through the broker cluster in connection with a live, deployed broker system operating in accordance with the publish/subscribe model in a distributed network or cloud computing environment. A test driver configured to receive a script causing errors to be simulated in association with one or more components in the messaging system is provided. A WAN simulator configured to simulate a network link and properties associated with the link is provided, with the properties including latency, jitter, bandwidth, and/or packet loss. The test driver receives a script. In response to the test driver receiving the script, errors in one or more components of the messaging system, once it has been deployed and gone live, are selectively generated in connection with at least one processor coordinating between respective actions for the test driver and the simulator in accordance with the script. With the aid of the at least one processor, it is determined whether the messaging system properly handled the selectively generated errors, e.g., in accordance with predefined load-balancing and/or failover policies.
In certain example embodiments, non-transitory computer readable storage media tangibly storing instructions that, when executed by at least one processor of a computer, may perform one of these and/or other methods.
These features, aspects, advantages, and example embodiments may be used separately and/or applied in various combinations to achieve yet further embodiments of this invention.